Range-Efficient Counting of Distinct Elements in a Massive Data Stream

Efficient one-pass estimation of F0, the number of distinct elements in a data stream, is a fundamental problem arising in various contexts in databases and networking. We consider rangeefficient estimation of F0: estimation of the number of distinct elements in a data stream where each element of the stream is not just a single integer but an interval of integers. We present a randomized algorithm which yields an ( , δ)-approximation of F0, with the following time and space complexities (n is the size of the universe of the items): (1) The amortized processing time per interval is O(log 1 δ log n ). (2) The workspace used is O( 1 2 log 1 δ logn) bits. Our algorithm improves upon a previous algorithm by Bar-Yossef, Kumar and Sivakumar [Proceedings of the 13th ACM–SIAM Symposium on Discrete Algorithms (SODA), 2002, pp. 623–632], which requires O( 1 5 log 1 δ log n) processing time per item. This algorithm can also be used to compute the max-dominance norm of a stream of multiple signals and significantly improves upon the previous best time and space bounds by Cormode and Muthukrishnan [Proceedings of the 11th European Symposium on Algorithms (ESA), Lecture Notes in Comput. Sci. 2938, Springer, Berlin, 2003, pp. 148–160]. This algorithm also provides an efficient solution to the distinct summation problem, which arises during data aggregation in sensor networks [Proceedings of the 2nd International Conference on Embedded Networked Sensor Systems, ACM Press, New York, 2004, pp. 250–262, Proceedings of the 20th International Conference on Data Engineering (ICDE), 2004, pp. 449–460].